How degenerate is the parametrization of neural networks with the ReLU activation function?

Neural network training is usually accomplished by solving a non-convex optimization problem using stochastic gradient descent. Although one optimizes over the networks parameters, the main loss function generally only depends on the realization of the neural network, i.e. the function it computes. Studying the optimization problem over the space of realizations opens up new ways to understand neural network training. In particular, usual loss functions like mean squared error and categorical cross entropy are convex on spaces of neural network realizations, which themselves are non-convex. Approximation capabilities of neural networks can be used to deal with the latter non-convexity, which allows us to establish that for sufficiently large networks local minima of a regularized optimization problem on the realization space are almost optimal. Note, however, that each realization has many different, possibly degenerate, parametrizations. In particular, a local minimum in the parametrization space needs not correspond to a local minimum in the realization space. To establish such a connection, inverse stability of the realization map is required, meaning that proximity of realizations must imply proximity of corresponding parametrizations. We present pathologies which prevent inverse stability in general, and, for shallow networks, proceed to establish a restricted space of parametrizations on which we have inverse stability w.r.t. to a Sobolev norm. Furthermore, we show that by optimizing over such restricted sets, it is still possible to learn any function which can be learned by optimization over unrestricted sets.Comment: Accepted at NeurIPS 201

An optimal control perspective on diffusion-based generative modeling

We establish a connection between stochastic optimal control and generative models based on stochastic differential equations (SDEs) such as recently developed diffusion probabilistic models. In particular, we derive a Hamilton-Jacobi-Bellman equation that governs the evolution of the log-densities of the underlying SDE marginals. This perspective allows to transfer methods from optimal control theory to generative modeling. First, we show that the evidence lower bound is a direct consequence of the well-known verification theorem from control theory. Further, we develop a novel diffusion-based method for sampling from unnormalized densities -- a problem frequently occurring in statistics and computational sciences.Comment: Accepted for oral presentation at NeurIPS 2022 Workshop on Score-Based Method

Liking and description of pasta sauces with varying mealworm content

Entomophagy is directly connected with culture, explaining why it is commonly rejected in Western countries. Due to increased meat consumption in recent years with its associated negative impacts on health and sustainability, the development of products based on alternative protein sources has become urgent. The larval form of Tenebrio molitor (mealworm) has the potential to substitute meat as it requires less resources and produces less emissions compared to other forms of meat production. Therefore, in this project we have aimed to develop pasta sauces with differing mealworm contents based on a common meat sauce and to test the acceptance with 91 consumers in Austria. Three sauces (100% mealworm, 50% mealworm and 50% meat, 100% meat) were developed and tested using a 9-point hedonic scale for acceptance, and the CATA (Check-All-That-Apply) method was integrated to also receive descriptive information. The analysis of the liking data revealed that the liking for the hybrid sauce with meat and mealworm content was comparable to the meat sauce (6.9 ± 1.8. vs. 6.5 ± 1.8, p > 0.05). Less liked was the sauce with the highest mealworm content (5.7 ± 1.8, p < 0.05). The CATA analysis demonstrated the strongest positive effects on the mean in terms of how much the products were liked for the attribute "fleshy" (0.8). On the other hand, the attributes "brownish" (-0.9) or "mushy" (-1.0) had the strongest negative effects on the mean of the liking of products. We have seen that meat cannot be substituted by mealworm immediately and completely. The results suggest a stepwise substitution and the further adaptation of products regarding the (negative and positive effecting) attributes to increase consumer acceptance

Group Testing for SARS-CoV-2 Allows for Up to 10-Fold Efficiency Increase Across Realistic Scenarios and Testing Strategies

Background: Due to the ongoing COVID-19 pandemic, demand for diagnostic testing has increased drastically, resulting in shortages of necessary materials to conduct the tests and overwhelming the capacity of testing laboratories. The supply scarcity and capacity limits affect test administration: priority must be given to hospitalized patients and symptomatic individuals, which can prevent the identification of asymptomatic and presymptomatic individuals and hence effective tracking and tracing policies. We describe optimized group testing strategies applicable to SARS-CoV-2 tests in scenarios tailored to the current COVID-19 pandemic and assess significant gains compared to individual testing. Methods: We account for biochemically realistic scenarios in the context of dilution effects on SARS-CoV-2 samples and consider evidence on specificity and sensitivity of PCR-based tests for the novel coronavirus. Because of the current uncertainty and the temporal and spatial changes in the prevalence regime, we provide analysis for several realistic scenarios and propose fast and reliable strategies for massive testing procedures. Key Findings: We find significant efficiency gaps between different group testing strategies in realistic scenarios for SARS-CoV-2 testing, highlighting the need for an informed decision of the pooling protocol depending on estimated prevalence, target specificity, and high- vs. low-risk population. For example, using one of the presented methods, all 1.47 million inhabitants of Munich, Germany, could be tested using only around 141 thousand tests if the infection rate is below 0.4% is assumed. Using 1 million tests, the 6.69 million inhabitants from the city of Rio de Janeiro, Brazil, could be tested as long as the infection rate does not exceed 1%. Moreover, we provide an interactive web application, available at , for visualizing the different strategies and designing pooling schemes according to specific prevalence scenarios and test configurations. Interpretation: Altogether, this work may help provide a basis for an efficient upscaling of current testing procedures, which takes the population heterogeneity into account and is fine-grained towards the desired study populations, e.g., mild/asymptomatic individuals vs. symptomatic ones but also mixtures thereof

The Regulatory Status Adopted by Lymph Node Dendritic Cells and T Cells During Healthy Aging Is Maintained During Cancer and May Contribute to Reduced Responses to Immunotherapy

Aging is associated with an increased incidence of cancer. One contributing factor could be modulation of immune cells responsible for anti-tumor responses, such as dendritic cells (DCs) and T cells. These immunological changes may also impact the efficacy of cancer immunotherapies in the elderly. The effects of healthy aging on DCs and T cells, and their impact on anti-mesothelioma immune responses, had not been reported. This study examined DCs and T cells in young (2-5 months; equivalent to 16-26 human years) and elderly (20-24 months; equivalent to 60-70 human years) healthy and mesothelioma-bearing C57BL/6J mice. During healthy aging, elderly lymph nodes adopted a regulatory profile, characterized by: (i) increased plasmacytoid DCs, (ii) increased expression of the adenosine-producing enzyme CD73 on CD11c+ cells, and (iii) increased expression of multiple regulatory markers (including CD73, the adenosine A2B receptor, CTLA-4, PD-1, ICOS, LAG-3, and IL-10) on CD8+ and CD4+ T cells, compared to lymph nodes from young mice. Although mesotheliomas grew faster in elderly mice, the increased regulatory status observed in healthy elderly lymph node DCs and T cells was not further exacerbated. However, elderly tumor-bearing mice demonstrated reduced MHC-I, MHC-II and CD80 on CD11c+ cells, and decreased IFN-? by CD8+ and CD4+ T cells within tumors, compared to young counterparts, implying loss of function. An agonist CD40 antibody based immunotherapy was less efficient at promoting tumor regression in elderly mice, which may be due to: (i) failure of elderly CD8+ T cells to up-regulate perforin, and (ii) increased expression of multiple regulatory markers on CD11c+ cells and T cells in elderly tumor-draining lymph nodes (including CD73, PD-1, ICOS, LAG-3, and TGF-ß). Our findings suggest that checkpoint blockade may improve responses to immunotherapy in elderly hosts with mesothelioma, and warrants further investigation